Projecting survey based estimates of 2022 onto small areas

Author

Affiliation

Simon Jackman

Honorary Professor, University of Sydney

1 Predicting vote with Census data on AES

How well can we predict vote in the AES data with predictors that are available on the CURF? This includes the SA4-level Census data predictors.

Brier score: 0.2467291

Confusion matrix: for each actual outcome (row), distribution of predictions assigned with max prob:

	Coalition	Greens	Independent	Labor	Other
Coalition	53.8	3.4	2.4	40.1	0.3
Greens	27.9	19.4	4.0	48.5	0.1
Independent	32.8	7.3	15.1	44.9	0.0
Labor	38.1	7.0	2.3	52.4	0.2
Other	51.5	5.0	2.6	39.9	1.0

1.1 Performance with modal missing data imputation

Brier score: 0.2454336

Confusion matrix: actual (rows) by predicted/max-prob (cols):

	Coalition	Greens	Independent	Labor	Other
Coalition	53.5	4.1	2.2	40.0	0.2
Greens	29.8	19.2	4.0	46.7	0.3
Independent	31.7	5.9	16.1	46.2	0.0
Labor	38.1	8.2	1.9	51.6	0.2
Other	54.3	5.2	3.4	35.3	1.7

2 Inference for populations

2.1 Vote marginal for the CURF data from imputed data

2022 House vote
Coalition	19.9
Greens	20.0
Independent	20.0
Labor	20.1
Other	20.0

2.2 Predict onto CURF from model fit to complete AES data

2022 House vote	MaxProb	SumProbs	Actual
Coalition	55.1	46.7	35.7
Greens	0.5	6.8	12.2
Independent	1.0	2.4	5.3
Labor	43.3	43.0	32.6

Variable importance:

rf variable importance

  only 20 most important variables shown (out of 28)

                                  Overall
RELP_collapsed                     100.00
AGE_continuous                      76.35
ANCP_collapse                       67.60
INCP_continuous                     52.93
MSTP                                44.60
UNI                                 37.24
Median_tot_hhd_inc_weekly_G02       36.17
Median_tot_fam_inc_weekly_G02       35.96
PC3                                 34.65
Median_mortgage_repay_monthly_G02   34.11
PC5                                 33.70
PC1                                 33.48
PC4                                 33.36
Median_tot_prsnl_inc_weekly_G02     33.34
PC6                                 32.91
PC9                                 32.72
Median_rent_weekly_G02              32.64
PC2                                 32.18
PC7                                 31.93
PC10                                31.89

3 Predictions for geographies

Combine predictions with other covariates on curf_complete

3.1 AREAENUM

Code

Inputs.table(transpose(yhat_areaenum_raw),
  {
    header: {
      Greens: "GRN",
      Independent: "IND"
    },
    
    format: {
      Coalition: x => d3.format(".1%")(x),
      Greens: x => d3.format(".1%")(x),
      Independent: x => d3.format(".1%")(x),
      Labor: x => d3.format(".1%")(x),
      Other: x => d3.format(".1%")(x)
    }
    
  }
)

3.2 Electoral divisions

Code

ced_compare = transpose(ced_compare_raw)
import {Plot} from "@mkfreeman/plot-tooltip"
Plot.plot({
  grid: false,
  width: 1020,
  height: 660,
  insetTop: 10,
  
  style: {
    fontSize: "16px"
  },
  
  x: {
    label: "Prediction →",
    labelOffset: 0
  },
  
  y: {
    label: "↑ 2022 Actual"
  },

  facet: {
    data: ced_compare,
    x: "facet_x",
    y: "facet_y"
    //y: "c200",
    //marginRight: 60 
    },
    
  fx: {
    padding: 0.1,
    labelOffset: 48,
    label: null,
    tickFormat: null
  },
  
  fy: {
    padding: 0.1,
    label: null,
    tickFormat: null
  }, 
  
  marks: [
  
    Plot.frame({stroke: "#aaa", strokeWidth: 0.5}),
  
    Plot.text(ced_compare,
      Plot.selectLast(
        {
          x: .4,
          y: .77,
          text: (d) => d.outcome
        }
        )
        ),
  
    Plot.link([1],
      {
        x1: 0, 
        x2: .8, 
        y1: 0, 
        y2: .8
      }
    ),
      
    Plot.dot(ced_compare,
    { 
      x: "estimate", 
      y: "actual" ,
      title: (d) => `${d.Division} \n 2022 actual: ${d3.format(".1%")(d.actual)} \n Predicted: ${d3.format(".1%")(d.estimate)}`,
      stroke: "#3333337f"
    }
    )
    
  ],
  
  tooltip: {
    fill: "red"
  }  
  
})

3.3 Demographic imbalance in CED extracts from CURF

Code

Plot.plot(
  {
   x: {
    label: "Total Variation Error, demographics →"
  },
  
  y: {
    label: "↑ Total Variation Error, 2022 House vote"
  },
  
  marks: [
    Plot.dot(
      transpose(ced_vote_tv_raw),
      {
        x: "tv.y",
        y: "tv.x",
        title: "Division"
      }
    )
  ]
  }
)

3.4 Rake CED extracts on demographics

Variation in weights:

Division	s	min	max
Bendigo	2.5780416	0.0000086	30.859540
Sydney	1.7987436	0.0009711	12.199306
Casey	1.7944043	0.0002709	12.687224
New England	1.6482850	0.0000509	17.086233
Flinders	1.6333033	0.0000658	19.850798
Mallee	1.5726479	0.0130783	14.372747
Moreton	1.5478647	0.0151816	11.127717
Boothby	1.4428878	0.0053361	14.322665
Bennelong	1.4376156	0.0006818	13.620257
Wannon	1.4066912	0.0143034	13.658697
Makin	1.4027926	0.0000106	14.575613
Oxley	1.3967175	0.0133082	15.856979
Durack	1.3734926	0.0123059	10.704866
Parkes	1.3263571	0.0074893	15.274519
Barker	1.3234352	0.0000471	9.226240
Menzies	1.3208845	0.0095473	9.279756
Fowler	1.2885169	0.0035197	13.186444
Barton	1.2850943	0.0040512	9.874167
Dickson	1.2760560	0.0067525	12.076173
Grey	1.2547192	0.0033467	14.347527
Macquarie	1.2526396	0.0103729	12.017586
Wentworth	1.2509479	0.0128664	9.629454
Blair	1.2316744	0.0066233	8.565736
Werriwa	1.2301238	0.0270912	13.247889
Nicholls	1.1973524	0.0167508	8.376280
McEwen	1.1939602	0.0166799	11.065251
Indi	1.1877036	0.0031423	8.382811
Brisbane	1.1436707	0.0156068	6.731995
Pearce	1.1387601	0.0001674	6.936223
Forde	1.1372234	0.0496546	9.618007
Hume	1.1190159	0.0000241	7.933744
Calare	1.0950057	0.0100608	8.436689
Griffith	1.0902510	0.0032716	7.620977
Greenway	1.0811338	0.0104492	8.453713
Whitlam	1.0467476	0.0582190	8.933270
Cook	1.0419148	0.0364373	13.953604
Isaacs	1.0404038	0.0515471	8.528984
Watson	1.0365405	0.0069620	7.264573
Bonner	1.0219053	0.0007176	7.441562
Deakin	1.0186199	0.0101797	8.220599
Gorton	0.9975589	0.0150294	7.033615
McMahon	0.9966654	0.0090440	12.839455
Paterson	0.9903204	0.0169473	7.453546
North Sydney	0.9830184	0.0185560	9.034169
Mitchell	0.9817589	0.0276072	14.058285
Groom	0.9693507	0.0232321	7.672426
Riverina	0.9450200	0.0000108	5.985934
Hawke	0.9406561	0.0061943	7.606924
Maranoa	0.9358403	0.0320753	6.922041
Curtin	0.9332948	0.0665137	13.835585
Farrer	0.9262207	0.0413802	11.565611
Kennedy	0.9247250	0.0304334	6.378256
Hughes	0.9232049	0.0578815	10.750154
Rankin	0.9084038	0.0562276	8.763248
Adelaide	0.8942120	0.0403152	8.575549
Cooper	0.8930565	0.0147946	7.515505
Hasluck	0.8903487	0.0243475	4.935279
Sturt	0.8860890	0.0516610	12.485082
Flynn	0.8845663	0.0626193	11.311120
Macnamara	0.8596823	0.0932197	12.305099
Perth	0.8448319	0.0685515	6.721007
Gellibrand	0.8408229	0.1001703	11.718316
Berowra	0.8393769	0.0115527	8.051217
Tangney	0.8261073	0.0653509	11.432890
Macarthur	0.8230161	0.0593629	8.172548
Reid	0.8225637	0.0335505	7.261453
Petrie	0.7908424	0.0546750	9.178553
Wright	0.7848379	0.0463837	7.367668
Hotham	0.7822530	0.0343599	7.895685
Jagajaga	0.7810443	0.0626807	8.612915
Shortland	0.7780678	0.0612036	6.363784
Cowan	0.7519233	0.0573316	7.946864
Gilmore	0.7496870	0.0460703	5.371107
Chisholm	0.7414970	0.0800713	9.205027
Capricornia	0.7389866	0.0773710	7.933174
La Trobe	0.7386513	0.0298348	5.699612
Fraser	0.7340315	0.0648693	7.765142
Lingiari	0.7265968	0.1286400	6.324772
Canning	0.7243790	0.0300809	8.691773
Higgins	0.7210573	0.0618029	8.158822
Canberra	0.7094029	0.1062587	7.947039
Robertson	0.7019808	0.0949866	7.162520
Scullin	0.7005307	0.1204179	6.758612
Lyne	0.7004547	0.0399576	5.005509
Calwell	0.6991125	0.1412705	5.234313
Wide Bay	0.6971233	0.0398779	6.112446
Clark	0.6962159	0.1053896	6.588164
Blaxland	0.6912721	0.0588989	4.951604
Eden-Monaro	0.6874472	0.1709286	6.638256
Richmond	0.6785678	0.0490977	7.124612
Kingsford Smith	0.6762163	0.1219288	5.377459
Mayo	0.6670176	0.0130771	5.563939
Bradfield	0.6619736	0.1106240	6.581924
Newcastle	0.6577251	0.1008045	6.896389
Corangamite	0.6542218	0.1132389	9.821724
Grayndler	0.6406635	0.0515023	4.828771
Solomon	0.6365387	0.0933097	4.986568
Lalor	0.6342810	0.1037005	5.545238
Cunningham	0.6319962	0.1236790	6.662259
Leichhardt	0.6312873	0.1206082	5.560562
Hinkler	0.6215269	0.0624515	5.085246
Moore	0.6209194	0.1144009	6.467165
Banks	0.6206930	0.1942334	5.672632
Chifley	0.6197328	0.0874513	4.782245
Holt	0.6164560	0.0443667	5.601250
Corio	0.6055126	0.1588281	6.255713
Franklin	0.5878195	0.1062300	6.417149
Melbourne	0.5733922	0.1660150	4.116145
Kooyong	0.5669183	0.1099543	5.828383
Moncrieff	0.5651720	0.1252925	5.681130
Swan	0.5607994	0.1452317	6.905182
Lindsay	0.5586542	0.1406972	4.154971
Lilley	0.5583978	0.1032221	5.358251
Parramatta	0.5563628	0.1227808	5.001099
Aston	0.5513989	0.1706816	5.040813
Monash	0.5465856	0.0877061	5.653150
Herbert	0.5452790	0.0868871	4.582953
Maribyrnong	0.5449552	0.1232340	5.488512
Forrest	0.5447834	0.0864451	6.175120
Dobell	0.5437386	0.1396497	4.395371
Dunkley	0.5435595	0.1713133	4.767757
Warringah	0.5434095	0.0987807	4.238104
Goldstein	0.5418253	0.1301611	5.525831
McPherson	0.5413583	0.1083691	5.434242
Longman	0.5353452	0.0768650	3.733408
Hunter	0.5295370	0.1100505	4.743364
Fremantle	0.5236029	0.1160798	4.401008
Bruce	0.5206726	0.1335883	5.451724
Ballarat	0.5202467	0.1180586	6.364325
O'Connor	0.5189334	0.0547496	4.075871
Cowper	0.5187695	0.0960574	4.770949
Dawson	0.5161077	0.0841633	6.524573
Gippsland	0.5136464	0.0846250	4.894813
Bowman	0.5133425	0.1172712	4.000088
Fadden	0.5016992	0.1358906	4.412945
Fairfax	0.5007181	0.0963938	3.927981
Fisher	0.4953689	0.1093310	3.648678
Braddon	0.4867593	0.0457050	5.209183
Wills	0.4777914	0.1836633	4.666492
Spence	0.4726178	0.1110867	3.321669
Brand	0.4682428	0.1037821	4.441956
Page	0.4636118	0.0689725	3.884404
Ryan	0.4635004	0.1900109	4.107098
Mackellar	0.4334901	0.0751803	2.521836
Kingston	0.4312959	0.1926938	3.132224
Burt	0.4157660	0.1990193	3.329439
Bass	0.4157223	0.2095118	3.090007
Bean	0.4075736	0.1933086	3.323672
Fenner	0.4057896	0.1337472	2.968078
Lyons	0.4017169	0.0543050	3.083122
Hindmarsh	0.3964710	0.1443560	3.695762

Weighted estimates of vote:

Warning in left_join(yhat %>% select(.id, outcome, p), ced_raked %>% unnest(d), : Detected an unexpected many-to-many relationship between `x` and `y`.
ℹ Row 1 of `x` matches multiple rows in `y`.
ℹ Row 223191 of `y` matches multiple rows in `x`.
ℹ If a many-to-many relationship is expected, set `relationship =
  "many-to-many"` to silence this warning.

Code

ced_compare_with_demo = transpose(ced_compare_with_demo_raw)
import {Plot} from "@mkfreeman/plot-tooltip"
Plot.plot({
  grid: false,
  width: 1020,
  height: 660,
  insetTop: 10,
  
  style: {
    fontSize: "16px"
  },
  
  x: {
    label: "Prediction →",
    labelOffset: 0
  },
  
  y: {
    label: "↑ 2022 Actual"
  },

  facet: {
    data: ced_compare_with_demo,
    x: "facet_x",
    y: "facet_y"
    //y: "c200",
    //marginRight: 60 
    },
    
  fx: {
    padding: 0.1,
    labelOffset: 48,
    label: null,
    tickFormat: null
  },
  
  fy: {
    padding: 0.1,
    label: null,
    tickFormat: null
  }, 
  
  marks: [
  
    Plot.frame({stroke: "#aaa", strokeWidth: 0.5}),
  
    Plot.text(ced_compare_with_demo,
      Plot.selectLast(
        {
          x: .4,
          y: .77,
          text: (d) => d.outcome
        }
        )
        ),
  
    Plot.link([1],
      {
        x1: 0, 
        x2: .8, 
        y1: 0, 
        y2: .8
      }
    ),
      
    Plot.dot(ced_compare_with_demo,
    { 
      x: "estimate", 
      y: "actual" ,
      title: (d) => `${d.Division} \n 2022 actual: ${d3.format(".1%")(d.actual)} \n Predicted: ${d3.format(".1%")(d.estimate)}`,
      stroke: "#3333337f"
    }
    )
    
  ],
  
  tooltip: {
    fill: "red"
  }  
  
})